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jl^  GAO 

^^^^^^^^^Accountability  *  Integrity  *  Reliability _ 

United  States  General  Accounting  Office 
Washington,  DC  20548 


August  29,  2002 

The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  W.  Warner 
Ranking  Minority  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Bob  Stump 
Chairman 

The  Honorable  Ike  Skelton 
Ranking  Minority  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Department  of  Defense  (DOD)  considers  the  transformation’  of  the 
U.  S.  military  a  strategic  imperative  to  meet  the  security  challenges  of  the 
new  century.  In  October  1998,  DOD  established  a  joint  concept 
development  and  experimentation  program  to  provide  the  engine  of 
change  for  this  transformation.  The  U.S.  Joint  Forces  Command,  the 
executive  agent  of  the  program,  carries  out  experiments — with  support 
from  the  military  services,  the  combatant  commands,  and  other  DOD 
organizations — ^to  explore  and  analyze  new  joint  operational  and 
organizational  concepts.  The  experiments  can  include  studies  and 
analyses,  conferences,  workshops,  seminars,  war  games,  and  simulated  or 
live  field  experiments. 

On  the  basis  of  these  experiments,  the  Joint  Forces  Command 
recommends  changes  aimed  at  achieving  significant  improvements  in  joint 


’  DOD  states  that  transformation  of  the  military  “results  from  the  exploitation  of  new 
approaches  to  operational  concepts  and  capabilities,  the  use  of  old  and  new  technologies, 
and  new  forms  of  organization  that  more  effectively  anticipate  new  or  still  emerging 
strategic  and  operational  challenges  and  opportunities  and  that  render  previous  methods  of 
conducting  war  obsolete  or  subordinate.” 
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capabilities.  The  Joint  Requirements  Oversight  Council  reviews  the 
Command’s  recommendations,  and  the  Chairman  of  the  Joint  Chiefs  of 
Staff  is  responsible  for  approving  them.  The  Council  also  oversees  the 
implementation  of  approved  recommendations.  The  cumulative  changes 
are  expected  to  contribute  to  the  way  U.S.  military  forces  are  transformed. 
In  November  2001,  DOD  also  created  an  Office  of  Force  Transformation  to 
foster  innovation  and  experimentation  and  assist  the  Secretary  of  Defense 
to  develop  guidance  and  identify  strategies  and  policies  for 
transformation. 

Because  of  the  central  role  DOD’s  joint  experimentation  program  plays  in 
military  transformation,  we  reviewed  related  activities  to  determine 
whether  changes  are  needed  to  improve  the  program’s  ability  to  focus  on 
future  joint  operations,  military  transformation,  and  the  rapid  integration 
of  results.  Specifically,  we  assessed  the  extent  to  which  (1)  the  Joint 
Forces  Command  obtains  participation  from  all  relevant  sources  in 
developing  and  conducting  its  experimentation  activities; 

(2)  the  recommendations  flowing  from  the  joint  experimentation  process 
have  been  approved  and  implemented;  and  (3)  the  necessary  management 
elements  are  in  place  to  foster  program  success.  We  are  providing  this 
report  to  you  because  of  your  oversight  responsibilities  and  interest  in 
military  transformation  issues. 


Results  in  Brief 


In  the  nearly  4  years  since  becoming  the  executive  agent  for  joint  concept 
development  and  experimentation,  the  Joint  Forces  Command  has 
increased  the  participation  of  key  DOD  stakeholders — the  military 
services,  the  combatant  commands,  and  other  organizations  and 
agencies — in  its  experimentation  activities.  The  Command  has  also 
expanded  the  participation  of  federal  agencies  and  departments, 
academia,  the  private  sector,  and  some  foreign  allies.  The  Command  uses 
many  different  ways  to  obtain  input — including  meetings,  seminars, 
workshops,  video  conferencing.  E-mail,  and  the  Internet — and  integrates 
the  results  of  military  operations,  technology  efforts,  and  other  DOD 
organizations’  experiments  into  its  activities.  Although  stakeholders 


^  The  Joint  Requirements  Oversight  Council  is  an  advisory  council  that  assists  the 
Chairman  of  the  Joint  Chiefs  of  Staff  in  identifying  and  assessing  the  priorities  for  joint 
military  requirements  to  achieve  current  and  future  military  capabilities.  Chaired  by  the 
Vice  Chairman  of  the  Joint  Chiefs  of  Staff,  the  Council  is  comprised  of  a  senior  officer  from 
each  of  the  military  services.  Representatives  from  other  DOD  entities,  such  as  the 
combatant  commands  and  the  Joint  Staff,  serve  in  am  advisory  role  to  the  Council. 
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determine  the  extent  of  their  involvement  in  joint  experimentation 
activities,  many  more  organizations  are  in  fact  involved  in  the  planning 
and  execution  of  this  year’s  major  field  experiment  than  there  were  in  a 
similar  experiment  2  years  ago.  Most  DOD  officials  whom  we  spoke  with 
were  generally  satisfied  with  the  way  the  Command  reached  out  to  obtain 
their  input,  but  many  also  believe  more  frequent  contacts  with  the  Joint 
Forces  Command  would  improve  communications.  Some  believe  that  the 
Command  could  do  more  to  expand  the  participation  of  non-DOD 
agencies  and  departments  and  allies  and  coalition  partners,  and  the 
Command  is  planning  initiatives  to  do  so,  though  this  outreach  would 
involve  broader  agency-resource  and  national-security  considerations. 

No  recommendations  flowing  from  joint  experimentation  have  been 
approved  or  implemented.  Although  the  Joint  Forces  Command  issued 
three  recommendations  nearly  a  year  ago,  they  were  not  approved  by  the 
Joint  Requirements  Oversight  Council  because  of  confusion  among  the 
Joint  Staff  and  the  Joint  Forces  Command  about  a  proposed  change  in 
guidance  that  required  additional  data  (on  costs  and  timelines)  be 
included  when  submitting  these  recommendations.  As  a  result,  it  is  not 
clear  when  these  recommendations  will  contribute  to  military 
transformation.  The  Command  plans  to  re-submit  the  recommendations 
with  the  required  data  later  this  year.  However,  at  the  time  we  concluded 
our  review,  the  official  guidance  on  what  information  should  accompany 
joint  experimentation  recommendations  had  not  been  approved  and 
issued.  In  addition,  although  it  is  too  early  to  assess  how  DOD  integrates 
the  results  of  its  experimentation  into  its  plans,  programs,  and  budgets, 
several  DOD  officials  expressed  concern  that  the  resource  allocation 
process  may  be  too  slow  to  provide  rapid  and  timely  funding  for  the 
implementation  of  new  concepts  emerging  from  joint  experimentation  and 
that  other,  more  rapid  resource  mechanisms  may  be  needed. 

While  DOD  has  been  providing  more  specific  and  clearer  guidance  for 
joint  experimentation,  DOD  and  the  Joint  Forces  Command  are  missing 
some  key  management  elements  that  are  generally  considered  necessary 
for  successful  program  management.  DOD  and  the  Joint  Forces  Command 
thus  lack  the  following  very  important  management  tools  needed  to  carry 
out  their  joint  experimentation  responsibilities: 

•  It  is  not  yet  clear  what  role  the  Office  of  Force  Transformation  will  have  in 
joint  experimentation  and  how  it  will  coordinate  its  activities  with  other 
DOD  stakeholders.  As  a  result,  the  Secretary  of  Defense  may  not  be 
effectively  using  this  new  office  to  interact  with  stakeholders  to  foster  and 
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monitor  joint  experimentation  as  a  means  to  advance  DOD’s 
transformation  agenda. 

•  DOD  has  not  developed  meaningful  performance  measures  to  assess  joint 
experimentation’s  contribution  to  the  advancement  of  military 
transformation.  Therefore,  DOD  cannot  gauge  the  progress  of  joint 
experimentation  efforts  and  of  the  program’s  overall  contribution  to 
transformation. 

•  The  Joint  Forces  Command  does  not  have  the  strategic-planning  tools 
(a  strategic  plan  with  associated  performance  plans  and  reports)  that 
would  allow  it  to  clearly  define  what  it  is  seeking  to  accomplish,  identify 
what  strategy  it  will  use  to  achieve  desired  results,  and  finally  measure 
how  well  it  is  succeeding  in  meeting  results-oriented  goals  and  objectives. 

We  are  making  recommendations  designed  to  improve  the  processes  and 
the  management  oversight  and  accountability  for  the  joint  concept 
development  and  experimentation  program  and  to  clarify  the  role  of  the 
Office  of  Force  Transformation.  In  its  comments  on  a  draft  of  this  report, 
DOD  agreed  with  our  recommendations  and  stated  that  our  findings 
provide  a  factual  summary  of  the  history  and  progress  of  joint 
experimentation  and  identify  the  critical  weaknesses  currently  being 
addressed  by  DOD.  It  expects  that  a  forthcoming  Transformation 
Planning  Guidance  or  subsequent  guidance  documents  will  address  the 
issues  and  problems  with  the  joint  experimentation  program  identified  in 
our  report. 


Background 


The  Joint  Forces  Command,  in  coordination  with  the  Joint  Staff,^  the 
services,  and  other  combatant  commands  and  DOD  agencies,  is 
responsible  for  creating  and  exploring  new  joint  war-fighting  concepts,  as 
well  as  for  planning,  designing,  conducting,  and  assessing  a  program  of 
joint  experimentation.  The  Command  executed  its  second  large-scale  field 
experiment.  Millennium  Challenge  2002,^  this  year,  and  it  plans  another 
one  in  2004  and  others  every  third  year  thereafter.  These  experiments  are 
intended  to  examine  how  well  the  concepts  previously  explored  by  the 


^  The  staff  that  assists  the  Chaimian  of  the  Joint  Chiefs  of  Staff  in  executing  the  Chairman’s 
responsibilities. 

Millennium  Challenge  2002  focuses  on  how  the  U.S.  military  can  use  its  current  and 
near-term  capabilities  differently  in  this  decade.  It  will  involve  the  U.S.  Special  Operations 
Command  and  forces  from  all  the  services.  Another  major  field  experiment — Olympic 
Challenge — is  planned  for  2004. 
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Command  in  smaller  venues  will  work  when  applied  with  the  emerging 
concepts  being  developed  by  the  services  and  other  combatant 
commands.  For  example,  Millennium  Challenge  2002  tested  how  well 
U.S.  forces  fared  against  a  regional  power  with  a  sizable  conventional 
military  force  and  so  called  “anti-access”  capabilities — which  can  include 
advanced  surface-to-air  missiles,  antiship  missiles  and  mines,  and 
chemical  and  biological  weapons — and  validated  the  results  of  earlier 
experiments  to  develop  the  Command’s  “rapid  decisive”  operations 
concept.  The  aim  of  the  experiment  was  to  come  up  with  changes  that  can 
be  made  during  the  current  decade.  (App.  1  provides  a  chronology  of  major 
events  important  to  joint  experimentation.) 

Over  the  next  several  years,  the  Command’s  experimentation  will  focus 
primarily  on  two  concepts:  one  to  develop  a  standing  joint  force 
headquarters  to  improve  joint  command  and  control;  another  to  conduct 
more  effective  joint  operations  through  “rapid  decisive”  operations. 

In  November  2001,  the  Chairman  of  the  Joint  Chiefs  of  Staff  directed  that 
the  Command  make  development  of  the  prototype  headquarters  its  highest 
near-term  priority.®  Additionally,  the  Command  will  develop  a  number  of 
other  concepts  aimed  at  specialized  issues  or  operational  problems  to 
support  the  two  primary  concepts. 

Joint  experimentation  is  a  continuous  process  that  begins  with  the 
development  of  new  operational  and  organizational  concepts  that  have  the 
potential  to  improve  significantly  joint  operations  (see  fig.  1).  The  Joint 
Forces  Command  identifies  new  joint  concepts  including  those  developed 
by  other  DOD  organizations  (such  as  the  Joint  Staff,  services,  and 
combatant  commands)  and  the  private  sector  and  tests  them  in 
experiments  that  range  from  simple  (workshops,  seminars,  war  games, 
and  simulations)  to  complex  (large-scale  virtual  simulations  and  “live” 
field  experiments).  Appendix  11  provides  additional  information  on  joint 
experimentation  program  activities. 


®  The  Unified  Command  Plan,  April  30,  2002,  assigns  the  Chairman  of  the  Joint  Chiefs  of 
Staff  responsibility  for  providing  overarching  guidance  on  joint  experimentation. 

®  The  Command  is  directed  to  develop  this  prototype  headquarters  by  2004  and  make  it 
available  for  implementation  by  the  regional  combatant  commands  during  fiscal  year  2005. 
The  headquarters  will  provide  uniform,  standard  operating  procedures,  tactics,  techniques, 
and  technical  system  requirements,  with  the  ability  to  move  expertise  among  commands. 
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Figure  1 ;  Joint  Concept  Deveiopment  and  Experimentation  Process 


New 

strategic  and 
operational 
challenges 

New 

military 

technological 

opportunities 


Revised  operational 
and  organizational  concepts 

Source:  Joint  Forces  Command. 


After  analyzing  experimentation  data,  the  Command  prepares  and  submits 
recommendations  to  the  Joint  Requirements  Oversight  Council  for  review 
and,  ultimately,  to  the  Chairman  of  the  Joint  Chiefs  of  Staff  for  approval/ 
Before  submitting  them  to  the  Council,  however,  the  Command  submits  its 
recommendations  to  the  Joint  Staff  for  preliminary  review  and 
coordination.  The  recommendations  are  distributed  for  review  and 
comment  to  the  Joint  Staff  directorates,  the  military  services,  the 
combatant  commands,  and  other  DOD  and  federal  government 
organizations.  The  Council  then  reviews  the  recommendations  and  advises 
the  Chairman  of  the  Joint  Chiefs  of  Staff  on  whether  they  should  be 
approved.  The  changes,  if  approved,  provide  the  basis  for  pursuing  the 
capabilities  needed  to  implement  a  specific  operational  concept. 

The  Council  is  also  responsible  for  overseeing  the  implementation  of  the 
recommendations,  but  it  can  designate  an  executive  agent,  such  as  the 
Joint  Forces  Command,  to  do  so.  The  Council  (or  its  designated  executive 


’’  We  recently  reported  on  the  status  of  actions  taken  by  the  Chairman  of  the  Joint  Chiefs  of 
Staff  to  improve  the  Joint  Requirements  Oversight  Council’s  oversight  and  the  joint 
requirements  process,  including  establishing  procedures  for  submitting  recommendations 
stemming  from  joint  experimentation  to  the  Council.  See  U.S.  General  Accounting  Office, 
Defense  Plans:  Status  of  DOD’s  Efforts  to  Improve  Its  Joint  Warfighting  Requirements 
Process,  GAO-02-100R  (Washington,  D.C.:  Nov.  9,  2001). 
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agent)  is  responsible  for  obtaining  the  resources  needed  to  implement  the 
recommendations  through  DOD’s  Planning,  Programming,  and  Budgeting 
System.®  The  Council  also  assists  the  Chairman,  in  coordination  with  the 
combatant  commands,  the  services,  and  other  DOD  organizations,  to 
identify  and  assess  joint  requirements  and  priorities  for  current  and  future 
military  capabilities.  The  Council  considers  requirements  (and  any 
proposed  changes)  for  joint  capabilities  across  doctrine,  organizations, 
training,  materiel,  leadership  and  education,  personnel,  and  facilities. 

The  Department  of  the  Navy’s  budget®  provides  funding  to  the  Joint  Forces 
Command  for  joint  experimentation  and  other  Command  missions. 

In  fiscal  year  2002,  the  Command  received  from  the  Navy  about 
$103  million  for  its  joint  concept  development  and  experimentation 
program,  and  it  planned  to  spend  about  half  of  this  amount  for  Millennium 
Challenge  2002.  The  Command  has  requested  that  the  Navy  provide  about 
$98  million  for  the  program  in  fiscal  year  2003.  The  Command  also 
provides  some  funds  to  the  services,  the  combatant  commands,  and  other 
DOD  organizations  for  efforts  that  support  its  program  activities. 

However,  the  services  fund  the  operations  and  support  costs  of  forces 
participating  in  joint  experimentation.  Also,  the  individual 
experimentation  efforts  of  the  services  and  the  combatant  commands  are 
funded  from  within  their  own  budgets. 


Participation  in  Joint 
Experimentation 
Continues  to  Evolve 


Since  it  first  began  joint  experimentation,  the  Joint  Forces  Command  has 
broadened  and  deepened  the  inclusion  of  other  DOD  organizations, 
federal  agencies  and  departments,  the  private  sector,  and  allies  and 
coalition  partners  in  its  process  for  capturing  and  identifying  new  joint 
ideas  and  innovations.  Organizations  participating  in  joint  experimentation 
are  generally  satisfied  with  current  opportunities  for  their  ideas  to  be 
considered,  and  many  have  increased  their  participation  in  the  program. 
However,  the  participation  of  different  stakeholders — the  extent  of  which 


®  The  Planning,  Programming,  and  Budgeting  System  is  designed  to  assist  the  Secretary  of 
Defense  in  making  choices  about  the  allocation  of  resources  among  a  number  of  competing 
or  possible  programs  and  alternatives  to  accomplish  specific  national  security  objectives. 
The  objective  of  the  system  is  to  provide  the  optimal  mix  of  forces,  equipment,  and  support 
that  can  be  achieved  within  fiscal  constraints. 

®  DOD  Directive  6100.3,  Support  of  the  Headquarters  of  Combatant  and  Subordinate  Joint 
Commands,  Nov.  15,  1999,  designates  the  Navy  as  the  executive  agent  responsible  for 
arranging  for  the  program-funding  of  the  Joint  Forces  Command’s  activities,  including  the 
joint  experimentation  program. 
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is  determined  by  the  stakeholder — ^varies  considerably  and  some  would 
like  more  visits  and  contacts  with  the  Command.  The  Command  is 
planning  initiatives  to  increase  stakeholder  participation  in  the  future, 
particularly  for  federal  agencies  and  departments  and  key  allies,  but  this 
increase  will  depend  on  agency-resource  and  national-security 
considerations. 


Participation  Has  Grown  As  the  program  gradually  evolved,  the  Joint  Forces  Command  solidified  a 
and  Continues  to  Expand  process  to  involve  the  military  services,  the  combatant  commands,  and 

other  DOD  organizations  in  the  planning  and  execution  of  its  joint 
experimentation  activities.  Because  future  joint  operations  will  involve 
diplomatic,  information,  and  economic  actions,  as  well  as  military 
operations,  many  DOD,  federal,  and  private  organizations  and 
governments  participate  and  provide  input  into  the  joint  experimentation 
program  (see  table  1).  The  Joint  Forces  Command  functions  as  a 
facilitator  to  solicit  and  coordinate  the  involvement  of  these  organizations 
and  incorporate  their  input,  as  appropriate,  into  concept  development  and 
experimentation  activities. 


Table  1:  Organizations  Involved  in  Joint  Experimentation 

Examples  of  organizations  and  government 

Organizationai  area 

entities 

DOD  headquarters  and  support 

Office  of  the  Secretary  of  Defense 

organizations 

Chairman  of  the  Joint  Chiefs  of  Staff  and  Joint  Staff 

Office  of  Force  Transformation 

Joint  Advanced  Warfighting  Program 

Defense  Inteiligence  Agency 

Nationai  Defense  University 

Miiitary  services 

Army 

Navy 

Air  Force 

Marine  Corps 

Combatant  commands 

U.S.  Centrai  Command 

U.S.  European  Command 

U.S.  Pacific  Command 

U.S.  Southern  Command 

U.S.  Special  Operations  Command 

U.S.  Transportation  Command 

U.S.  Space  Command 

U.S.  Strategic  Command 

Federal  agencies  and 

Department  of  State 

departments 

Department  of  Justice 

Centrai  Inteiligence  Agency 

Private  sector 

Academia 

Think  tank  groups 

Industry 
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Organizational  area 

Examples  of  organizations  and  government 
entities 

Allies  and  potential  coalition 
partners 

•  Some  North  Atlantic  Treaty  Organization  countries 

•  Australia 

Source:  GAO  interviews  and  DOD  data. 

Because  the  stakeholders  determine  the  extent  of  their  participation  in  the 
program,  it  can  vary  considerably.  However,  Joint  Forces  Command 
officials  stated  that  participation  by  the  services,  the  combatant 
commands,  and  other  DOD  organizations  has  grown  steadily  since  the 
program  was  created  and  continues  to  grow,  as  participants  become 
increasingly  aware  of  the  strong  emphasis  that  DOD  leaders  are  placing  on 
experimentation.  For  example,  in  contrast  to  the  first  field  experiment 
in  2000,  which  had  limited  involvement  by  the  services,  this  year’s 
Millennium  Challenge  has  seen  the  services  more  actively  involved  in  early 
planning,  and  their  individual  experiments  better  coordinated  and 
integrated  into  the  field  experiment.  Our  comparison  of  participation  in 
the  Command’s  major  field  experiment  in  2000  with  plans  for  this  year’s 
experiment  found  a  significant  increase  in  the  diversity  and  number  of 
participating  organizations  and  in  the  number  of  concepts  and  initiatives 
proposed  by  these  organizations.  For  example,  the  total  number  of 
organizations  participating  in  Millennium  Challenge  2002  more  than 
doubled  from  the  prior  experiment  in  2000  (from  12  to  29  organizations), 
and  the  total  number  of  service  initiatives  increased  from  4  to  29. 

The  Command  provides  several  ways  for  organizations  to  participate  and 
provide  inputs:  they  can  review  program  plans  and  strategies;  attend 
meetings,  seminars,  and  workshops;  take  part  in  experimentation 
activities;  and  use  various  communication  tools  such  as  E-mail,  Internet, 
and  video  conferencing.  Additionally,  the  Command  obtains  input  from  the 
various  experimentation  and  research  and  development  organizations  of 
the  military  services  and  of  some  combatant  commands  and  DOD 
organizations. “  The  Command  also  considers  the  results  of  Advanced 


Participating  organizations  would  include  each  of  the  services’  experimentation 
programs,  service  battle  labs  and  supporting  research  and  development  organizations, 
DOD  organizations  such  as  Defense  Advanced  Research  Projects  Agency  and  Joint 
Advanced  Warfighting  Program,  and  federally  funded  research  and  development 
organizations  such  as  the  Institute  for  Defense  Analyses,  the  RAND  Corporation,  and  the 
Center  for  Naval  Analysis. 
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Concept  Technology  Demonstrations  efforts, “  innovations,  and  recent 
military  operations  in  developing  its  program.  For  example,  as  a  result  of 
its  operational  experiences  in  Kosovo,  the  U.S.  European  Command 
identified  various  joint  capability  shortfalls  in  its  recent  list  of  Command 
priorities  as  a  means  of  guiding  the  Joint  Forces  Command  in  selecting 
focal  areas  and  activities  for  experimentation.  Further,  the  Command  is 
taking  steps  to  (1)  align  its  experimentation  activities  with  the  schedules 
of  major  service  and  combatant  command  exercises  and  (2)  adjust  its 
program  to  allow  for  earlier  consideration  of  new  concepts  proposed  by 
the  services  and  the  combatant  commands  in  the  input  process.  These 
adjustments  would  improve  synchronization  of  experiments  with  the 
availability  of  forces  and  the  training  schedules  of  the  services  and  the 
combatant  commands,  allow  for  greater  involvement  of  these  entities  in 
the  process,  and  increase  the  likelihood  that  joint  requirements  are 
sufficiently  considered  early  in  the  development  of  concepts. 

Participating  organizations  also  provide  input  during  the  annual 
preparation  of  two  key  joint  experimentation-program  documents: 
the  Chairman  of  the  Joint  Chiefs  of  Staffs  guidance  on  joint 
experimentation  and  the  Joint  Forces  Command’s  Joint  Concept 
Development  and  Experimentation  Campaign  Plan^^  (see  fig.  2).  Each 
year  the  Chairman  provides  guidance  to  the  Joint  Forces  Command  to  use 
in  developing  its  Campaign  Plan  for  joint  concept  development  and 
experimentation.  The  basis  for  the  Chairman’s  guidance  is  derived  from 
several  sources,  including  strategy  and  planning  documents,  studies,  and 
other  assessments.^^  Additionally,  key  DOD  stakeholders,  including  the 
Chairman’s  Joint  Warfighting  Capability  Assessment  teams^^  and  the  Joint 


DOD  initiated  the  Advanced  Concept  Technology  Demonstration  program  to  determine 
the  extent  to  which  a  given  mature  technology  will  improve  military  capabilities  before 
entering  the  normal  acquisition  process  and,  by  using  mature  technology,  reduce  the  length 
of  time  to  develop  and  acquire  weapon  systems. 

The  Campaign  Plan  provides  the  structure  for  executing  experimentation  and  outlines 
the  Joint  Forces  Command’s  objectives,  methods,  and  events  to  achieve  those  objectives. 
The  most  recent  Campaign  Plan  is:  U.S.  Joint  Forces  Command,  The  Joint  Concept 
Development  and  Eocperimentation  Campaign  Plan  Fiscal  Years  2002-2007  (Norfolk, 

Va.;  2002). 

Examples  of  these  sources  include  the  President’s  National  Security  Strategy,  the 
Secretary  of  Defense’s  annual  defense  planning  guidance,  and  the  Chairman  of  the  Joint 
Chief  of  Staffs  Joint  Vision  and  National  Military  Strategy. 

Teams  of  DOD  war-fighting  and  functional-area  experts  appraise  various  aspects  of  joint 
war  fighting  within  their  respective  areas.  The  Chairman  of  the  Joint  Chiefs  of  Staff  uses 
the  teams’  assessments  to  develop  guidance  and  to  support  decision  making. 
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Requirements  Oversight  Council,  provide  input  to  the  Joint  Staff  to  use  in 
developing  the  Chairman’s  guidance.  The  Joint  Forces  Command  uses  this 
guidance,  with  additional  input  from  DOD  stakeholders,  in  preparing  its 
Campaign  Plan,  which  is  the  primary  vehicle  for  synchronizing  its  joint 
experimentation  activities  and  coordinating  resources.  The  Command  also 
solicits  and  considers  input  for  the  Campaign  Plan  from  some  other 
federal  agencies  and  departments,  academia,  private  sector,  and  allies. 
After  review  and  endorsement  by  the  combatant  commands,  the  services, 
and  the  Joint  Requirements  Oversight  Council,  the  Chairman  approves  the 
Campaign  Plan. 
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Figure  2:  Involvement  of  Organizations  in  Preparing  Joint  Experimentation 
Guidance  and  Campaign  Plan 


Approve  Joint 
Experimentatbn 
Campaign  Pian 


Source:  Joint  Staff  and  Joinf  Forces  Command. 


Stakeholders  Generally  Officials  at  the  military  services,  the  combatant  commands,  and  other 
Satisfied  but  Point  to  Some  DOD  organizations  we  talked  with  said  they  were  generally  satisfied  with 
Possible  Improvements  opportunities  for  input  provided  by  the  Joint  Forces  Command.  At  the 

same  time,  DOD  stakeholders  have  taken  various  actions  to  increase  their 
participation.  Some,  however,  would  like  more  contacts  and 
communication  with  the  Command.  The  Command  is  responding  with 
some  initiatives. 
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Each  service,  the  Joint  Staff,  the  U.S.  Special  Operations  Command,  the 
U.S.  Space  Command,  as  well  as  some  DOD  and  federal  agencies  (such  as 
the  National  Imagery  and  Mapping  Agency  and  the  National  Security 
Agency)  have  assigned  liaison  officers  at  the  Joint  Forces  Command/'^ 
However,  officials  at  the  Central,  Pacific,  and  Southern  Commands  stated 
that  their  staffing  levels  currently  do  not  allow  them  to  devote  personnel 
in  this  role.  Combatant  command  officials  indicated  that  the  frequency  and 
number  of  meetings,  conferences,  and  other  events  held  at  the  Joint 
Forces  Command  often  make  it  difficult  for  their  organizations  to  attend. 
The  officials  believe  that  as  a  result,  the  views  and  positions  of  their 
organizations  are  not  always  fully  captured  in  some  discussions  and 
deliberations.  Some  of  the  combatant  commands  have  or  are  planning  to 
establish  their  own  joint  experimentation  offices.  Officials  from  the  Pacific 
and  Special  Operations  Commands  stated  that  although  their  respective 
joint  experimentation  offices  are  largely  focused  on  supporting  their  own 
experimentation  efforts,  the  offices  provide  a  cadre  of  staff  who  can  better 
coordinate  and  participate  more  consistently  in  the  Joint  Forces 
Command’s  joint  experimentation  program.  For  example.  Pacific 
Command  officials  said  that  their  own  experimentation  efforts  to  improve 
the  command  of  joint  operations  over  the  past  few  years  have  contributed 
to  joint  experimentation  by  providing  significant  insights  for  the  Joint 
Forces  Command’s  development  of  the  standing  joint-force  headquarters 
concept.  Central  Command  and  Southern  Command  officials  said  their 
Commands  have  plans  to  establish  similar  offices  soon. 

While  satisfied  with  their  participation  and  their  ability  to  provide  input 
into  the  program,  officials  at  some  combatant  commands  believe  that  a 
number  of  things  could  be  done  to  improve  the  program,  assuming 
resources  are  available.  They  believe  that  the  Joint  Forces  Command 
could  increase  its  visits  to  and  participation  in  combatant-command 
activities.  Some  of  the  officials  also  believe  that  if  the  Joint  Forces 
Command  assigned  liaison  officers  to  their  commands,  the  Command 
would  obtain  first-hand  knowledge  and  a  better  appreciation  of  the 
various  commands’  individual  requirements.  These  officials  believe  that 
such  a  presence  at  their  commands  would  demonstrate  the  Joint  Forces 
Command’s  commitment  to  joint  experimentation  and  would  allow  for 
interaction  with  staff  throughout  their  commands.  The  Joint  Forces 
Command  does  not  favor  doing  this  because  of  the  cost  and  the  difficulty 


Additionally,  Germany,  the  United  Kingdom,  Canada,  and  Australia  have  each  assigned  a 
liaison  officer  at  the  Joint  Forces  Command. 
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in  providing  the  staff  necessary  to  fulfill  this  role.  Officials  at  the  Pacific, 
Central,  and  Southern  Commands  also  believe  that  some  level  of  funding 
should  be  provided  to  the  combatant  commands  for  their  use  in 
supporting  individual  command  and  the  Joint  Forces  Command 
experimentation  efforts.  Combatant  command  officials  stated  that 
currently,  funds  from  other  command  activities  must  be  diverted  to 
support  these  efforts. 

Out  of  concern  about  the  need  to  improve  communications  and 
participation  in  joint  experimentation  planning,  the  Joint  Forces 
Command  is  planning  some  initiatives  such  as  the  following: 

•  It  plans  to  create  a  virtual  planning-center  site  for  joint  experimentation  on 
its  Intranet  to  provide  DOD  stakeholders  with  easily  accessible  weekly 
updates  to  information  on  planned  experiments;  participants;  goals  and 
objectives;  and  ongoing  experimentation  by  the  Joint  Forces  Command, 
the  services,  the  combatant  commands,  and  DOD  agencies.  It  plans  to 
develop  the  requirements  for  the  site  during  fall  2002  and  to  initiate  the 
project  soon  after. 

•  It  established  Project  Alpha — a  “think-tank”  group — in  early  2002  to 
provide  another  source  of  input  and  outputs.  The  project  will  interface 
with  researchers  throughout  DOD,  Department  of  Energy  national 
laboratories,  private  industry,  and  academia  to  find  cutting-edge 
technologies  for  inclusion  in  service  and  joint  experimentation. 

This  relationship  will  provide  an  opportunity  for  the  Joint  Forces 
Command  to  leverage  the  work  of  these  organizations  and  similarly,  for 
these  organizations  to  gain  a  better  understanding  of  and  include  their 
work  in  the  joint  experimentation  program. 


Expanding  Interagency 
and  Foreign  Participation 
in  Experimentation  Events 
Is  Being  Considered 


As  the  joint  experimentation  program  matured,  participation  by  non-DOD 
federal  agencies  and  departments  gradually  increased.  Participation, 
however,  depends  upon  the  agencies’  desire  to  be  involved  and  their 
available  resources.  Lack  of  involvement  could  lead  to  missed 
opportunities.  And  participation  by  allies  and  coalition  partners  has  been 
limited  by  security  concerns. 


The  Joint  Forces  Command’s  input  process  allows  individual  federal 
agencies  and  departments,  such  as  the  Departments  of  State  and  Justice, 
to  participate  in  joint  experimentation  events  as  they  choose.  Interagency 
participation  is  improving,  according  to  Command  officials.  For  example, 
federal  agencies  and  departments  are  participating  in  Millennium 
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Challenge  2002  to  assist  the  Command  in  developing  its  standing  joint- 
force  headquarters  concept. 

However,  resource  and  staffing  constraints  prevent  some  agencies  and 
departments  from  taking  part  in  experiments.  For  example,  according  to  a 
Joint  Forces  Command  official,  the  Department  of  Transportation  and  the 
Central  Intelligence  Agency  decided  not  to  send  representatives  to 
Millennium  Challenge  2002  because  of  staffing  constraints.  Not  only  could 
non-DOD  agencies  provide  important  insights  and  contributions  to  joint 
operations,  but  also  some  important  opportunities  could  be  missed  if  these 
agencies  do  not  consistently  participate  in  joint  experimentation  events. 

While  federal  agencies  and  departments  are  beginning  to  increase  their 
role  in  joint  experimentation,  several  service  and  combatant  command 
officials  we  spoke  with  believe  that  greater  involvement  is  needed  because 
of  the  role  these  organizations  are  likely  to  have  in  future  joint  operations. 
For  example,  these  non-DOD  federal  agencies  and  departments  would 
provide  support  (economic,  diplomatic,  and  information  actions)  to 
U.S.  military  forces  in  their  conduct  of  operations  aimed  at  defeating  an 
adversary’s  war-making  capabilities — support  that  is  critical  to 
implementation  of  the  Joint  Forces  Command’s  rapid  decisive  operations 
concept. 

Several  DOD  (service,  combatant  command.  Office  of  the  Secretary  of 
Defense,  and  other  DOD  organizations)  officials  we  spoke  with  believe 
that  the  Joint  Forces  Command  should  explore  ways  to  boost  the 
participation  and  involvement  of  allies  and  coalition  partners  in  joint 
experimentation.  Joint  Forces  Command  officials  agree  and  believe  that 
such  cooperation  would  foster  a  better  understanding  of  allied 
perspectives,  allow  the  Command  to  leverage  concept  development  work, 
expand  available  capabilities,  and  facilitate  the  development  of 
multinational  capabilities.  The  Command  recently  created  a  multinational 
concept-development  and  experimentation  site  on  its  Intranet  to  facilitate 
the  involvement  of  allies  and  coalition  partners  in  joint  experimentation. 
However,  some  DOD  officials  believe  that  the  Joint  Forces  Command 
should  do  more  because  future  U.S.  military  operations  will  likely  be 
conducted  with  other  countries.  The  officials  stress  that  other  nations’ 
military  personnel  should  be  included  in  experiments  to  develop  new 
operational  concepts,  if  these  concepts  are  to  be  successful.  Joint  Forces 
Command  officials  pointed  out,  however,  that  the  participation  and 
involvement  of  other  countries  are  often  constrained  by  restrictions  on 
access  to  sensitive  security  information.  For  example.  North  Atlantic 
Treaty  Organization  countries  only  participated  as  observers  in 
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Millennium  Challenge  2002  because  of  security  information  restrictions. 
The  Command,  however,  plans  to  develop  ways  to  better  handle  these 
restrictions  to  allow  greater  participation  by  other  nations  in  its  next 
major  field  experiment  in  2004. 


No  Recommendations 
for  Change  Have  Been 
Approved  or 
Implemented 


Nearly  4  years  after  the  program  was  established,  only  three 
recommendations  have  flowed  from  the  joint  experimentation  program, 
and  none  of  them  have  been  approved.  Confusion  about  proposed  changes 
in  guidance  regarding  the  information  required  for  submitting  these 
recommendations  has  partly  delayed  their  approval.  At  the  time  we 
concluded  our  review,  official  guidance  on  what  information  should 
accompany  joint  experimentation  recommendations  had  not  been 
approved.  In  addition,  several  DOD  officials  expressed  concern  that  the 
process  used  to  review  and  approve  recommendations,  the  same  as  that 
used  for  major  acquisition  programs,  may  not  be  the  most  appropriate  for 
a  program  whose  aim  is  to  integrate  changes  quickly.  However,  the 
officials  could  not  pinpoint  any  specific  impasses  in  the  approval  process. 
The  DOD  officials  are  also  concerned  about  potential  delays  in  the 
integration  of  new  concepts  because  of  the  lengthy  DOD  resource 
allocation  process. 


Concerns  Raised  about  the  The  Joint  Forces  Command  submitted  one  recommendation  to  the 
Delays  in  Approving  Chairman  of  the  Joint  Chiefs  of  Staff  in  August  2001  and  two  more  in 

Recommendations  November  2001  (see  table  2).  At  the  time  we  ended  our  review,  none  of  the 

recommendations  had  been  approved.  The  recommendations  to  improve 
the  planning  and  decision-making  capabilities  of  joint  forces  and  provide 
better  training  for  personnel  conducting  theater  missile  defense 
operations  were  based  on  analyses  of  results  of  experiments  carried  out  in 
the  first  3  years  of  joint  experimentation.  Inputs  included  two  major 
experiments:  Millennium  Challenge  2000  (live  field  experiment  in 
August-September  2000)  and  the  Unified  Vision  2001  (virtual  simulation 
experiment  in  May  2001).  The  first  recommendation  was  submitted  for 
review  just  3  months  after  the  end  of  the  last  experiment. 
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Table  2:  First  Three  Recommendations  Submitted  by  Joint  Forces  Command 

Date  submitted 

Recommendation 

Description 

Changes  required" 

for  review 

Collaborative 

Develop  deployable 

• 

Joint  doctrine 

August  18,  2001 

environment  and 

Internet-like  tools  to 

• 

Training 

collaborative  tools 

improve  the 

• 

Leadership  and 

collaborative  planning 

education 

and  decision-making 
capabilities  of  joint 
forces. 

• 

Personnel 

Joint  intelligence 

Develop  objectives  to 

• 

Joint  doctrine 

November  1 , 

preparation  of  the 

train  and  exercise  joint 

• 

Training 

2001 

battle  space 

forces  on  intelligence 

• 

Leadership  and 

requirements  for 
preparing  the  battle 
space,  with  a  focus  on 
theater  missile  defense. 

education 

Theater  missile 

Develop  a  training 

• 

Training 

November  1 , 

defense  and  time- 

course  to  improve  the 

• 

Leadership  and 

2001 

critical  targeting 

effectiveness  of  teams 

education 

team  training 

involved  in  theater 

• 

Personnel 

missile  defense  and 
time-critical  targeting 
activities. 

• 

Facilities 

‘‘Recommendations  can  propose  changes  be  made  to  doctrine,  organizations,  training,  materiei, 
ieadership  and  education,  personnei,  and/or  faciiities. 


Source:  Joint  Forces  Command. 


According  to  a  Joint  Staff  official,  however,  approval  of  the 
recommendations  has  been  delayed  because  Joint  Forces  Command  and 
Joint  Staff  officials  were  confused  about  proposed  changes  in  guidance. 

In  May  2001,  the  Joint  Requirements  Oversight  Council  proposed  new 
guidance,  which  would  require  that  information  on  costs  and  timelines  be 
included  in  joint  experimentation  recommendations.  Prior  guidance  did 
not  require  such  information.  Although  the  recommendations  went 
through  preliminary  review  by  the  Joint  Staff,  the  omission  was  not  caught 
until  the  recommendations  were  to  be  scheduled  for  review  by  the  Joint 
Requirements  Oversight  Council.  Joint  Forces  Command  officials  told  us 
that  they  were  not  aware  of  the  change  in  guidance  until  that  time.  When 
we  ended  our  review.  Joint  Forces  Command  officials  were  working  with 
the  Joint  Staff  to  assess  how  much  data  could  be  prepared  and  when. 
Command  officials  said  that  the  recommendations  will  be  resubmitted  in 
fall  2002  together  with  other  recommendations  emerging  from  Millennium 
Challenge  2002.  As  a  result,  no  recommendations  have  yet  been  reviewed 
or  approved.  Also,  at  the  time  we  ended  our  review,  the  draft  guidance  on 
joint  experimentation  recommendations  had  not  been  approved  and 
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issued.  This  guidance  will  become  especially  important  because  joint 
experimentation  is  expected  to  produce  new  recommendations  more 
rapidly  as  the  program  matures. 

The  requirement  for  costs  and  timeline  data  is  consistent  with  that  of 
recommendations  for  major  weapon-system-acquisition  programs. 
However,  joint  experimentation  officials  at  the  Joint  Forces  Command 
believe  that  requiring  this  type  of  information  on  joint-experimentation 
recommendations  may  not  be  appropriate  because  (1)  these 
recommendations  are  generally  intended  to  convince  decision  makers  to 
develop  particular  joint  capabilities,  not  specific  weapon  systems; 

(2)  the  new  requirement  may  slow  the  preparation  of  future 
recommendations;  and  (3)  it  will  be  difficult  to  provide  accurate  estimates 
of  costs  and  timelines  for  recommendations  that  span  further  into  the 
future.  It  is  too  early  to  determine  whether  these  concerns  are  valid. 


Concerns  Raised  about  the 
Process  for  Funding 
Implementation  of 
Recommendations 


Some  DOD  officials  were  also  concerned  that  the  system  currently  used  to 
allocate  resources  to  implement  joint-experimentation 
recommendations — DOD’s  Planning,  Programming,  and  Budgeting 
System — may  not  be  the  most  efficient  because  it  usually  takes  a  long  time 
to  review,  approve,  and  provide  funding  in  future  budgets. 

A  recommendation  approved  in  2002,  for  example,  would  not  be 
incorporated  into  DOD’s  budget  until  2004  or  even  later.  This  delay  could 
result  in  missed  opportunities  for  more  rapid  implementation. 


A  Joint  Staff  official  told  us  that  the  Joint  Staff  and  the  Joint  Forces 
Command  recently  adjusted  the  timing  of  events  to  better  align  the  joint 
experimentation  process  with  the  Planning,  Programming,  and  Budgeting 
System.  Additionally,  DOD  established  a  special  fund  for  the  Joint  Forces 
Command  to  use  as  a  temporary  funding  source  to  speed  up  the 
implementation  of  certain  critical  or  time-sensitive  recommendations. 

This  source  will  provide  early  funding  for  implementation  until  funding  is 
provided  through  DOD’s  Planning,  Programming,  and  Budgeting  System. 
However,  Joint  Forces  Command  and  other  DOD  officials  believe  other 
ways  to  implement  new  joint  capabilities  within  the  framework  of  existing 
budget  and  oversight  practices  may  need  to  be  considered. 
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Some  Key 
Management 
Elements  Are  Missing 


DOD  has  been  providing  more  specific  and  clearer  guidance  on  its  goals, 
expectations,  and  priorities  for  the  joint  experimentation  program. 
Nevertheless,  the  management  of  joint  experimentation  is  missing  a 
number  of  key  elements  that  are  necessary  for  program  success:  some 
roles  and  responsibilities  have  not  yet  been  defined;  current  performance 
measures  are  not  adequate  to  assess  progress;  and  the  Joint  Forces 
Command  lacks  strategic  planning  tools  for  the  program. 


Guidance  More  Clearly  DOD  officials  stated  that  the  joint  experimentation  program  had  difficulty 

Defined  it®  first  years  because  guidance  was  evolving  and  was  not  specific: 

DOD’s  transformation  goals  were  not  adequately  hnked  to  transformation 
efforts,  and  roles  and  responsibilities  were  not  clearly  defined.'®  Over  time, 
the  Secretary  of  Defense  and  the  Chairman  of  the  Joint  Chiefs  of  Staff 
have  provided  more  specific  guidance  on  the  goals  and  expectations  for 
joint  experimentation  and  its  contribution  to  DOD’s  transformation 
efforts.  Guidance  for  joint  experimentation  has  evolved  gradually  over  the 
program’s  nearly  4-year  life  span,  partly  because  of  shifting  defense 
priorities  and  lack  of  clarity  about  the  roles  of  various  DOD  stakeholders. 
Roles  and  responsibilities  have  also  matured  with  the  program."^ 

The  Secretary  of  Defense’s  2001  Quadrennial  Defense  Review  Reporf^ 
established  six  transformation  goals,  which  include  improving 
U.S.  capabilities  to  defend  the  homeland  and  other  bases  of  operations, 
denying  enemies  sanctuary,  and  conducting  effective  information 
operations.  According  to  DOD  officials,  the  Secretary  of  Defense’s  most 
recent  planning  guidance  tasked  the  Joint  Forces  Command  to  focus  its 
experimentation  on  developing  new  joint  operational  concepts  for  these 
goals.  To  begin  meeting  these  goals,  the  Chairman  has  also  provided  the 
Joint  Forces  Command  with  clarifying  guidance  that  identified  specific 
areas  for  the  Command  to  include  in  its  experimentation,  such  as  the 


We  have  pointed  this  difficulty  out  in  prior  transformation-related  reports.  See  U.S. 
General  Accounting  Office,  Military  Transformation:  Navy  Efforts  Should  Be  More 
Integrated  and  Focused,  GAO-01-853  (Washington,  D.C.:  Aug.  2,  2001)  and  U.S.  General 
Accounting  Office,  Military  Transformation:  Army  Has  A  Comprehensive  Plan  for 
Managing  Its  Transformation  but  Faces  Major  Challenges,  GAO-02-96  (Washington,  D.C.: 
Nov.  16,  2001). 

The  roles  and  responsibilities  for  most  DOD  organizations  are  broadly  defined  in  the 
Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3010. 02A,  Joint  Vision  Implementation 
Master  Plan,  Apr.  15,  2001. 

Department  of  Defense,  Quadrennial  Defense  Review  Report  (Washington,  D.C.:  2001). 
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development  of  a  standing  joint-force  headquarters  concept  and  of  a 
prototype  to  strengthen  the  conduct  of  joint  operations.  The  Command 
has  reflected  this  new  guidance  in  its  latest  Joint  Concept  Development 
and  Experimentation  Campaign  Plan.  Additionally,  the  Secretary  of 
Defense  reassigned  the  Command’s  geographic  responsibilities  to  focus  it 
more  clearly  on  its  remaining  missions,  particularly  transformation  and 
joint  experimentation. 

DOD  officials  at  both  headquarters  and  the  field  believe  that  the  recent 
guidance  begins  to  provide  a  better  framework  for  the  Joint  Forces 
Command  to  establish  and  focus  its  joint  experimentation  efforts. 

Some  officials,  however,  believe  that  future  guidance  should  further  clarify 
the  link  between  joint  experimentation  and  DOD  priorities  and  the 
required  resources  necessary  to  support  joint  experimentation.  DOD,  in  its 
comments  to  a  draft  of  this  report,  stated  that  it  expects  the 
Transformation  Planning  Guidance — currently  being  prepared  by  the 
Office  of  the  Secretary  of  Defense — will  establish  the  requirements 
necessary  to  link  experimentation  to  changes  in  the  force. 


Role  of  Office  of  Force  While  roles  and  responsibilities  for  DOD  organizations  are  now  broadly 

Transformation  Is  Not  defined,  the  new  DOD  Office  of  Force  Transformation’s  role  in  joint 

Clear  experimentation  and  its  relationship  to  other  stakeholders  have  not  yet 

been  clearly  established.  The  Office’s  charter  or  terms  of  reference  have 
not  been  released.  DOD  plans  to  issue  a  directive  later  this  year  that  will 
include  a  charter  and  description  of  the  Office’s  authorities  and 
responsibilities.  However,  there  is  still  uncertainty  about  the  extent  of 
authority  and  involvement  the  Office  will  have  in  the  joint  experimentation 
program  and  the  Office’s  ability  to  link  the  program  with  DOD’s  overall 
transformation  efforts.  Joint  Forces  Command  and  other  DOD  officials 
consider  having  a  transformation  advocate  in  the  Office  of  the  Secretary  of 
Defense  as  a  beneficial  link  between  the  Joint  Forces  Command’s,  the 
services’,  and  the  combatant  commands’  joint  experimentation  programs 
and  DOD’s  overall  transformation  agenda. 

According  to  DOD’s  2001  Quadrennial  Defense  Review  Report,  the  Office 
of  Force  Transformation,  created  in  November  2001,  is  to  play  a  role  in 
fostering  innovation  and  experimentation  and  should  have  an  important 
responsibility  for  monitoring  joint  experimentation  and  for  providing  the 
Secretary  of  Defense  with  policy  recommendations. 

An  Office  of  Force  Transformation  official  told  us  that  the  Office  will  be  an 
advocate  for  transformation  and  will  help  develop  guidance  and  make 
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recommendations  on  transformation  issues  to  the  Secretary  of  Defense 
(the  Office  provided  comments  on  the  Secretary’s  annual  planning 
guidance  and  developed  instructions  for  the  services  on  preparing  their 
first  transformation  road  maps). 

The  Office  has  also  decided  to  take  a  cautious  approach  in  carrying  out  its 
mission  because  of  possible  resistance  from  other  DOD  organizations,  the 
same  official  said.  The  Office  plans  to  offer  its  assistance  to  DOD 
organizations  in  their  transformation  efforts  and  attempt  to  influence  their 
thinking  on  key  issues,  rather  than  asserting  itself  directly  into  their 
efforts,  for  example  by  funding  military  use  of  existing  private-sector 
technology  to  act  as  a  surrogate  for  evaluating  possible  concepts,  uses, 
and  designs. 

Joint  Forces  Command  officials  stated  that  as  of  May  2002,  they  had  had 
only  limited  discussions  with  the  Office  and  had  not  established  any 
working  agreements  on  how  the  Office  would  participate  in  the  joint 
experimentation  program. 

The  Office  of  Force  Transformation  has  only  recently  assembled  its  staff 
and  is  beginning  to  plan  its  work  and  establish  contacts  within  DOD  and 
with  other  organizations.  The  Office’s  budget  for  fiscal  years  2002  and  2003 
is  about  $18  million  and  $35  million,  respectively. 


Current  Performance  DOD’s  performance  measures  (or  metrics)  for  assessing  joint 

Measures  Are  Not  a  Useful  experimentation — by  measuring  only  the  number  of  experiments  carried 

Assessment  Tool  provide  a  meaningful  assessment  of  the  program’s 

contribution  toward  meeting  its  performance  goal  for  military 
transformation  because  they  are  only  quantitative.  Consistent  with  good 
management  practices  and  in  order  to  effectuate  the  purposes  of  the 
Government  Performance  and  Results  Act  of  1993,“  federal  agencies 
devise  results-oriented  metrics  that  provide  an  assessment  of  outcomes  or 
the  results  of  programs  as  measured  by  the  difference  they  make.  In  its 
fiscal  year  2000  performance  report,^”  the  most  recent  it  has  issued,  DOD 
described  the  performance  indicators  for  the  joint  experimentation 


“  Pub.L.  103-62,  sec.  2(b),  Aug.  3,  1993,  sometimes  referred  to  as  the  Results  Act. 

Department  of  Defense,  Government  Performance  and  Results  Act:  Department  of 
Defense  Fiscal  Year  2000  Performance  Report  (Washington,  D.C.:  2001).  As  of  August 
2002,  DOD  did  not  submit  a  performance  report  to  the  Congress  for  fiscal  year  2001. 
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program  in  terms  of  the  number  of  experiments  conducted  against  a  target 
goal  for  the  prior,  current,  and  following  fiscal  years.  In  fiscal  year  2000, 
DOD  exceeded  its  target  number  of  experiments  and  did  not  project  any 
shortfalls  in  meeting  its  target  in  the  next  fiscal  year.  Although  this 
measure  does  provide  a  quantitative  assessment  of  experimental  activity, 
it  does  not  provide  a  meaningful  method  for  assessing  how  joint 
experimentation  is  helping  to  advance  military  transformation. 

An  Office  of  the  Secretary  of  Defense  official  stated  that  DOD  recognizes 
that  better  performance  measures  are  needed  for  assessing  how  joint 
experimentation  advances  transformation  and  for  two  other  metrics 
currently  used  to  assess  its  military  transformation  goal.^^  The  official 
stated  that  developing  such  measures  is  a  challenge  because  joint 
experimentation  does  not  easily  lend  itself  to  traditional  measurement 
methods.  For  example,  most  programs  consider  a  failure  as  a  negative 
event,  but  in  joint  experimentation,  a  failure  can  be  considered  as  a 
success  if  it  provides  insights  or  information  that  is  helpful  in  evaluating 
new  concepts  or  the  use  of  new  technologies. 

An  Office  of  the  Secretary  of  Defense  official  told  us  that  the  RAND 
Corporation  and  the  Institute  for  Defense  Analyses  recently  completed 
studies  to  identify  possible  performance  measures  for  assessing  the 
progress  of  transformation.  DOD  is  evaluating  them  and  is  preparing  the 
Transformation  Planning  Guidance  to  provide  more  specific  information 
on  the  priorities,  roles,  and  responsibilities  for  executing  its 
transformation  strategy.  The  same  official  stated  that  the  new  guidance 
will  include  a  discussion  of  the  types  of  performance  measures  needed  for 
assessing  transformation  progress  or  will  assign  an  organization  to 
determine  them.  In  either  case,  measures  will  still  need  to  be  developed 
and  implemented.  DOD  plans  to  issue  the  new  guidance  later  in  2002  but 
has  not  determined  how  new  performance  measures  would  be 
incorporated  into  its  annual  performance  report. 


Joint  Forces  Command’s 
Program  Lacks  Strategic 
Planning  Tools 


The  Joint  Forces  Command  has  not  developed  the  strategic  planning 
tools — a  strategic  plan,  an  associated  performance  plan,  and  performance¬ 
reporting  tools — for  assessing  the  performance  of  the  joint 


DOD’s  fiscal  year  2000  performance  report  included  three  metrics  for  assessing  its 
transformation  of  U.S.  military  forces  performance  goal:  joint  experimentation, 
procurement  spending,  and  defense  technology  objectives. 
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experimentation  program.  Strategic  planning  is  essential  for  this  type  of 
program,  especially  considering  its  magnitude  and  complexity  and  its 
potential  implications  for  military  transformation.  Such  planning  provides 
an  essential  foundation  for  defining  what  an  organization  seeks  to 
accomplish,  identifies  the  strategy  it  will  use  to  achieve  desired  results, 
and  then  determines — ^through  measurement — how  well  it  is  succeeding  in 
reaching  results-oriented  goals  and  achieving  objectives.  Developing 
strategic-planning  tools  for  the  joint  experimentation  program  would  also 
be  consistent  with  the  principles  set  forth  in  the  Government  Performance 
and  Results  Act  of  1993,  which  is  the  primary  legislative  framework  for 
strategic  planning  in  the  federal  government. 

The  Joint  Forces  Command  prepares  an  annual  Joint  Concept 
Development  and  Experimentation  Campaign  Plan  that  broadly 
describes  the  key  goals  of  its  program,  the  strategy  for  achieving  these 
goals,  and  the  planned  activities.  However,  a  February  2002  progress 
report,  prepared  by  the  Joint  Forces  Command’s  Joint  Experimentation 
Directorate,  on  the  development  of  the  Directorate’s  performance 
management  system  indicated  that  one-fourth  of  those  organizations 
providing  feedback^^  on  the  Campaign  Plan  believed  that  the  Plan  lacks 
specificity  in  terms  of  the  program’s  goals  and  objectives  and  an 
associated  action  plan  that  outlines  the  activities  to  be  carried  out  in  order 
to  achieve  those  goals.  Officials  we  spoke  with  at  the  military  services,  the 
combatant  commands,  and  the  Joint  Forces  Command  all  cited  the  need 
for  more  specific  and  clearer  goals,  objectives,  and  performance  measures 
for  the  program. 

In  the  progress  report,  the  Command  acknowledged  the  benefits  of 
strategic  planning  and  the  use  of  this  management  tool  to  align  its 
organizational  structure,  processes,  and  budget  to  support  the 
achievement  of  missions  and  goals.  The  report  proposed  that  the 
Command  develop  a  strategic  plan,  possibly  by  modifying  its  annual 
Campaign  Plan,  and  subsequently  prepare  a  performance  plan  and  a 
performance  report.  Command  officials  indicated  that  the  basic 
requirements  of  a  strategic  plan  could  be  incorporated  into  the 
Campaign  Plan,  but  they  were  unsure,  if  such  an  approach  were  taken, 
whether  the  changes  could  be  made  before  the  annual  Campaign  Plan  is 


DOD  organizations  providing  comments  to  the  draft  Campaign  Plan  included  various 
offices  and  activities  of  the  Army,  the  Navy,  the  Air  Force,  the  Marine  Corps,  the 
U.S.  Special  Operations  Command,  the  U.S.  Strategic  Command,  the  U.S.  Southern 
Command,  the  U.S.  Pacific  Command,  and  the  U.S.  European  Command. 
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issued  later  this  year.  Similarly,  the  Joint  Forces  Command  has  had 
difficulty  in  developing  specific  performance  measures  for  joint 
experimentation.  A  Command  official  stated  that  the  Command  has  tried 
to  leverage  the  performance  measures  developed  by  other  organizations 
like  itself,  but  found  that  there  is  widespread  awareness  throughout  the 
research  and  development  community,  both  within  and  outside  DOD,  that 
such  measures  are  needed  but  do  not  exist.  Additionally,  a  Joint  Forces 
Command  official  stated  that  whatever  metrics  the  Command  develops 
must  be  linked  to  its  mission-essential  tasks  for  joint  experimentation  and 
that  the  Command  is  currently  developing  these  tasks.  At  the  time  we 
ended  our  review,  the  Command  had  identified  six  broad  areas  for  which 
specific  metrics  need  to  be  developed.  These  included  quality  of  life, 
customer  relationships,  and  experimentation  process  management. 


Conclusions 


After  nearly  4  years,  the  Joint  Forces  Command’s  process  for  obtaining 
inputs  for  the  development  and  execution  of  DOD’s  joint  experimentation 
program  has  become  more  inclusive.  However,  questions  continue  about 
whether  the  program  is  the  successful  engine  for  change  envisioned  when 
it  was  established.  Since  the  program’s  inception,  only  three 
recommendations  have  flowed  from  experimentation  activities  and  their 
review,  approval,  and  implementation  have  been  delayed  from  confusion 
over  a  change  in  guidance  that  required  additional  information  be  included 
in  the  recommendations.  As  a  result,  no  recommendations  for  change  have 
been  approved  or  implemented  to  date.  To  the  extent  that  the  draft 
guidance  on  what  should  be  submitted  with  joint  experimentation 
recommendations  can  be  officially  approved  and  issued,  future 
recommendations  could  be  submitted  for  approval  and  implementation 
more  quickly.  Underscoring  the  need  to  finalize  the  guidance  is  the 
anticipated  recommendations  to  be  made  after  this  year’s  major  field 
experiment.  Millennium  Challenge  2002. 

The  lack  of  strategic  planning  for  joint  experimentation  deprives  the 
Joint  Forces  Command  of  necessary  tools  to  effectively  manage  its 
program.  Implementation  of  strategic  planning  at  the  Joint  Forces 
Command  would  create  a  recurring  and  continuous  cycle  of  planning, 
program  execution,  and  reporting  and  establish  a  process  by  which  the 
Command  could  measure  the  effectiveness  of  its  activities  as  well  as  a 
means  to  assess  the  contributions  of  those  activities  to  the  operational 
goals  and  mission  of  the  program.  Such  planning  could  also  provide  a 
tool — one  that  is  currently  missing — to  identify  strengths  and  weaknesses 
in  the  development  and  execution  of  the  program  and  a  reference 
document  for  the  effective  oversight  and  management  of  the  program. 
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Performance  measures  developed  under  the  Command’s  strategic 
planning  could  provide  the  standard  for  assessing  other  experimentation 
efforts  throughout  DOD,  which  are  also  lacking  such  metrics. 

The  lack  of  a  meaningful  performance  measure  for  assessing  the 
contribution  of  the  joint  experimentation  program  to  advance 
DOD’s  transformation  agenda  limits  the  usefulness  and  benefit  of  this 
management  tool  to  assist  congressional  and  DOD  leaders  in  their 
decision-making  responsibilities.  Establishing  a  “meaningful”  joint 
experimentation  performance  measure  for  its  annual  performance  report 
would  provide  congressional  and  DOD  leadership  a  better  assessment  of 
the  program’s  contribution  and  progress  toward  advancing 
transformation.  Such  a  metric  would  also  be  consistent  with  the  intent  of 
the  Results  Act  to  improve  the  accountability  of  federal  programs  for 
achieving  program  results.  Because  the  role  and  relationships  of  the 
Secretary  of  Defense’s  new  Office  of  Force  Transformation  have  not  yet 
been  clarified,  the  Secretary  may  not  be  effectively  using  this  office  in 
DOD’s  transformation  efforts.  This  office,  if  given  sufficient  authority, 
could  provide  the  Secretary  with  a  civilian  oversight  function  to  foster  and 
monitor  the  joint  experimentation  program  to  ensure  that  it  is  properly 
supported  and  provided  resources  to  advance  the  DOD’s  overall 
transformation  agenda.  Rectifying  these  shortcomings  is  critical  in  view  of 
the  importance  that  DOD  has  placed  on  joint  experimentation  to  identify 
the  future  concepts  and  capabilities  for  maintaining  U.S.  military 
superiority. 


Recommendations  for 
Executive  Action 


To  improve  the  management  of  DOD’s  joint  experimentation  program,  we 
recommend  that  the  Secretary  of  Defense  direct  the  Chairman  of  the  Joint 
Chiefs  of  Staff  to 


•  approve  and  issue  guidance  that  clearly  defines  the  information  required 
to  accompany  joint  experimentation  recommendations  for  the  Joint 
Requirements  Oversight  Council’s  review  and  approval  and 

•  require  the  Commander  in  Chief  of  the  U.S.  Joint  Forces  Command  to 
develop  strategic  planning  tools  to  use  in  managing  and  periodically 
assessing  the  progress  of  its  joint  experimentation  program. 

We  further  recommend  that  the  Secretary  of  Defense 

•  develop  both  quantitative  and  qualitative  performance  measures  for  joint 
experimentation  in  DOD’s  annual  performance  report  to  provide  a  better 
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assessment  of  the  program’s  contribution  to  advancing  military 
transformation  and 

•  clarify  the  role  of  the  Office  of  Force  Transformation  and  its  relationship 
to  the  Chairman  of  the  Joint  Chiefs  of  Staff,  the  Joint  Forces  Command, 
and  other  key  DOD  stakeholders  in  DOD’s  joint  experimentation  program. 


Agency  Comments 
and  Our  Evaluation 


We  received  written  comments  from  DOD  on  a  draft  of  this  report,  which 
are  included  in  their  entirety  as  appendix  111.  DOD  agreed  with  our 
recommendations  and  indicated  that  it  expects  that  a  forthcoming 
Transformation  Planning  Guidance  and  subsequent  guidance  will  be 
responsive  to  them  by  clarifying  roles  and  missions  across  DOD, 
implementing  recommendations  for  changes,  and  establishing  clear 
objectives.  We  believe  such  strategic  guidance  from  the  Secretary  of 
Defense  could  provide  a  significant  mechanism  for  better  linking  and 
clarifying  the  importance  of  the  joint  experimentation  program  with 
DOD’s  transformation  agenda.  DOD  also  provided  technical  comments  to 
the  draft  that  were  incorporated  in  the  report  where  appropriate. 


Scope  and 
Methodology 


To  determine  the  extent  to  which  the  Joint  Forces  Command  obtains  input 
from  stakeholders  and  other  relevant  sources  in  developing  and 
conducting  its  joint  experimentation  activities,  we  reviewed  an  array  of 
documents  providing  information  about  participants  in  joint 
experimentation,  including  guidance  and  other  policy  documents,  position 
papers,  fact  sheets,  reports,  and  studies  of  the  military  services,  the 
combatant  commands,  the  Joint  Staff,  and  other  DOD  organizations. 

We  also  reviewed  Joint  Forces  Command  plans  and  reports.  Additionally, 
we  made  extensive  use  of  information  available  on  public  and  DOD 
Internet  web  sites.  To  assess  the  change  in  participation  by  various 
stakeholders  over  time,  we  compared  the  differences  in  the  numbers  of 
participating  organizations  and  initiatives  provided  by  these  organizations 
between  the  Joint  Forces  Command’s  first  two  major  field  experiments  in 
2000  and  2002  (Millennium  Challenge  2000  and  Millennium  Challenge 
2002).  We  conducted  discussions  with  officials  at  five  combatant 
commands,  the  Joint  Staff,  the  military  services,  and  other  DOD 
organizations,  such  as  the  Joint  Advanced  Warfighting  Program  and  the 
Defense  Advanced  Research  Projects  Agency.  Appendix  IV  lists  the 
principal  organizations  and  offices  where  we  performed  work.  At  the 
Joint  Forces  Command,  we  discussed  with  joint  experimentation  officials 
the  process  for  soliciting  and  incorporating  inputs  for  joint 
experimentation  from  the  military  services  and  the  combatant  commands. 
We  also  attended  conferences  and  other  sessions  hosted  by  the 
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Joint  Forces  Command  to  observe  and  learn  about  joint  experimentation 
participants  and  their  contributions  and  coordination.  For  example,  we 
attended  sessions  for  the  Command’s  preparation  of  its  annual  Joint 
Concept  Development  and  Experimentation  Campaign  Plan  and 
planning  for  this  year’s  Millennium  Challenge  experiment.  With  officials 
from  each  of  the  services  and  the  combatant  commands,  we  discussed 
perceptions  of  the  effectiveness  of  coordination  and  participation  in  joint 
experimentation.  We  also  obtained  observations  about  participants’ 
involvement  from  several  defense  experts  who  track  joint  experimentation 
and  military  transformation.  Although  we  did  not  include  a  specific 
assessment  of  the  individual  experimentation  efforts  of  the  services  and 
combatant  commands,  we  did  discuss  with  service  and  command  officials 
how  their  efforts  were  coordinated  and  integrated  into  joint 
experimentation.  We  also  did  not  determine  the  extent  that  individual 
inputs  obtained  from  various  participating  organizations  were  considered 
and  incorporated  into  the  joint  experimentation  program. 

To  determine  the  extent  to  which  recommendations  flowing  from  the  joint 
experimentation  process  have  been  approved  and  implemented,  we 
reviewed  and  analyzed  data  that  tracked  the  progress  of  the  first  three 
joint  experimentation  recommendations  submitted  by  the  Joint  Forces 
Command.  We  also  obtained  and  analyzed  relevant  guidance  and  held 
discussions  with  Joint  Staff,  Joint  Forces  Command,  and  Office  of  the 
Secretary  of  Defense  officials  on  the  Joint  Requirements  Oversight  Council 
process  for  reviewing  and  approving  joint  experimentation 
recommendations.  We  also  discussed  issues  relating  to  implementation  of 
joint  experimentation  recommendations  through  DOD’s  Planning, 
Programming,  and  Budgeting  System. 

To  assess  whether  key  management  elements,  such  as  policy, 
organization,  and  resources,  were  in  place  for  the  program,  we  conducted 
a  comprehensive  review  of  current  legislative,  policy,  planning,  and 
guidance  documents  and  reports  and  studies.  We  used  the  principles  laid 
out  in  the  Government  Performance  and  Results  Act  of  1993  as  an 
additional  benchmark  for  assessing  the  adequacy  of  performance 
measures  established  for  the  program  and  of  tools  used  to  manage  the 
program.  We  also  discussed  the  status  and  evolution  of  joint 
experimentation  oversight  and  management,  including  office  roles  and 
responsibilities  and  joint  experimentation  metrics,  with  officials  at  the 
Joint  Forces  Command,  the  Joint  Staff,  the  services,  the  combatant 
commands,  the  Office  of  the  Secretary  of  Defense,  the  Office  of  Force 
Transformation,  and  other  DOD  organizations.  Several  defense  experts 
who  follow  joint  experimentation  and  military  transformation  discussed 
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with  us  joint  experimentation  oversight  and  management  and  gave  us  their 
impressions  regarding  current  joint  experimentation  management 
practices. 

Our  review  was  conducted  from  October  2001  through  May  2002  in 
accordance  with  generally  accepted  government  auditing  standards. 


We  are  sending  copies  of  this  report  to  interested  congressional 
committees,  the  Secretary  of  Defense,  the  Chairman  of  the  Joint  Chiefs  of 
Staff,  and  the  Commander  in  Chief,  U.S.  Joint  Forces  Command.  We  will 
also  make  copies  available  to  others  upon  request.  In  addition,  this  report 
will  be  available  at  no  charge  on  the  GAO  Web  site  at  http://www.gao.gov. 

Please  contact  Richard  G.  Payne  at  (757)  552-8119  if  you  or  your  staff  have 
any  questions  concerning  this  report.  Key  contacts  and  contributors  to  this 
report  are  listed  in  appendix  V. 


Henry  L.  Hinton,  Jr. 

Managing  Director 

Defense  Capabihties  and  Management 
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Appendix  I:  Chronology  of  Key  Events 
Important  to  Joint  Experimentation 


Date 

Event 

Relevance  to  joint  experimentation 

May  1 996 

Chairman  of  the  Joint  Chiefs  of  Staff 
issued  Joint  Vision  2010. 

This  vision  of  future  war  fighting  provides  a  conceptual  template  for 
the  Department  of  Defense’s  (DOD)  transformation  efforts  across  aii 
elements  of  the  armed  forces. 

May  1997 

□CD’s  Report  of  the  Quadrenniai 
Defense  Review  issued. 

Report  discussed  the  importance  of  preparing  for  future  national 
security  challenges.  It  concluded  that  DOD  needed  to  institutionalize 
innovative  investigations,  such  as  war-fighting  experiments,  to 
ensure  future  concepts  and  capabiiities  are  successfuily  integrated 
into  the  forces  in  a  timely  manner. 

May  1 998 

Secretary  of  Defense  designated 
Commander  in  Chief,  U.S.  Joint 

Forces  Command, “  as  executive  agent 
for  joint  experimentation. 

The  Secretary  of  Defense  tasked  the  Joint  Forces  Command  to 
design  and  conduct  joint  war-fighting  experimentation  to  explore, 
demonstrate,  and  evaluate  joint  war-fighting  concepts  and 
capabilities. 

April  1998 

Joint  Advanced  Warfighting  Program 
estabiished. 

DOD  established  the  program  at  the  Institute  for  Defense  Analyses  to 
serve  as  a  catalyst  for  achieving  the  objectives  of  Joint  Vision  2010 
(and  later  Joint  Vision  2020).  To  that  end,  the  program  is  to  develop 
and  explore  breakthrough  operational  concepts  and  capabilities  that 
support  DOD’s  transformation  goais. 

October  1 998 

Joint  Forces  Command  assumed 
responsibiiity  as  the  executive  agent 
for  joint  experimentation. 

Joint  concept  development  and  experimentation  program  initiated. 

June-August  1999 

Joint  Advanced  Warfighting  Program 
conducted  the  first  joint  experiment  for 
Joint  Forces  Command. 

An  experiment — J9901 — that  investigated  approaches  for  attacking 
critical  mobile  targets.  Experiment  aiiowed  the  Joint  Forces 

Command  to  begin  its  iearning  process  on  how  to  conduct  joint 
experimentation. 

September  1999 

Report  of  the  Defense  Science  Board 
Task  Force  on  DOD  Warfighting 
Transformation  issued. 

Report  proposed  severai  recommendations  to  promote  military 
transformation. 

June  2000 

Chairman  of  the  Joint  Chiefs  of  Staff 
issued  Joint  Vision  2020. 

Updated  vision  statement  described  the  joint  war-fighting  capabiiities 
required  through  2020. 

August-September 

2000 

Miliennium  Chalienge  2000  major  fieid 
experiment  conducted. 

The  first  major  field  experiment  coordinated  by  the  Joint  Forces 
Command  among  the  services  and  other  stakeholders. 

April  2001 

Chairman  of  the  Joint  Chiefs  of  Staff 
issued  updated  Joint  Vision 
Implementation  Master  Plan. 

Guidance  described  the  process  for  generation,  coordination, 
approval,  and  implementation  of  recommendations  emerging  from 
joint  experimentation  and  defined  the  roies  and  responsibiiities  of 

DOD  stakehoiders. 

April  2001 

Transformation  Study  Report: 
Transforming  Military  Operational 
Capabilities  issued. 

Study  conducted  for  the  Secretary  of  Defense  to  identify  capabilities 
needed  by  U.S.  forces  to  meet  the  twenty-first  century  security 
environment.  Made  several  recommendations  directed  at  improving 
joint  experimentation. 

May  2001 

Joint  Forces  Command  conducted 
Unified  Vision  2001  experiment. 

A  major  joint  experiment — largeiy  modeling  and  simuiation — 
conducted  to  refine  and  explore  several  war-fighting  concepts,  such 
as  “rapid  decisive”  operations. 

August  2001 

Secretary  of  Defense’s  planning 
guidance  issued. 

Required  studies  by  defense  agencies  and  the  Joint  Staff  to  develop 
transformation  road  maps  and  a  standing-joint-force  headquarters 
prototype. 

September  2001 

DCD’s  Quadrennial  Defense  Review 
Report  issued. 

The  report  established  priorities  and  identified  major  goals  for 
transforming  the  Armed  Forces  to  meet  future  challenges.  It  called 
for  new  operational  concepts,  advanced  technological  capabilities, 
and  an  increased  emphasis  on  joint  organizations,  experimentation, 
and  training. 
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Appendix  I:  Chronology  of  Key  Events 
Important  to  Joint  Experimentation 


Date 

Event 

Relevance  to  joint  experimentation 

November  2001 

Chairman  of  the  Joint  Chiefs  of  Staff 
issued  joint  experimentation  guidance. 

The  guidance  directed  the  Joint  Forces  Command  to  focus  its  near- 
term  experimentation  on  developing  a  standing  joint  force 
headquarters  prototype. 

November  2001 

Office  of  Force  Transformation 
established. 

Office  assists  the  Secretary  of  Defense  in  identifying  strategy  and 
policy,  and  developing  guidance  for  transformation. 

April  2002 

Unified  Command  Rian  2002  issued. 

Plan  reduced  the  number  of  missions  assigned  to  the  Joint  Forces 
Command  to  allow  the  Command  to  devote  more  attention  to  its 
remaining  missions  such  as  ioint  experimentation. 

May  2002 

Secretary  of  Defense’s  planning 
guidance  issued. 

The  guidance  directed  the  Joint  Forces  Command  to  develop  new 
joint  concepts  that  focus  on  the  six  transformation  goals  set  forth  in 
the  2001  Quadrenniai  Defense  Review  Report. 

July-August  2002 

Joint  Forces  Command  conducted 
Millennium  Challenge  2002. 

Second  major  field  experiment  conducted  to  culminate  a  series  of 
experiments  to  assess  “how”  to  do  rapid  decisive  operations  in  this 
decade. 

■  In  October  1999,  the  U.S.  Atlantic  Command  was  redesignated  as  the  U.S.  Joint  Forces  Command. 
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Appendix  II;  Additional  Information  on  Joint 
Concept  Development  and  Experimentation 
Activities 


The  Joint  Forces  Command  uses  various  types  of  assessment  activities  to 
develop,  refine,  and  validate  joint  concepts  and  associated  capabilities. 

As  shown  in  figure  3,  the  Command  begins  to  move  through  the  five  joint 
concept  development  phases  by  conducting  workshops,  seminars,  and  war 
games  to  develop  information  and  identify  possible  areas  to  explore  in 
developing  new  concepts  and  associated  capabilities  and  then  uses 
simulated  or  live  experiment  events  to  confirm,  refute,  or  modify  them. 
These  activities  vary  in  scale  and  frequency,  but  each  activity  becomes 
larger  and  more  complex.  They  can  involve  a  small  group  of  retired  flag 
officers  and  academics,  up  to  100  plarmers,  operators,  and  technology 
experts,  or  several  thousand  in  the  field.  Near  the  end  of  the  process,  the 
Command  will  conduct  a  large-scale  simulation  experiment  (such  as 
Unified  Vision  2001),  followed  by  a  major  field  experiment  (such  as 
Millennium  Challenge  2002).  The  process  continuously  repeats  itself  to 
identify  additional  new  concepts  and  capabihties. 


Figure  3:  Joint  Concept  Deveiopment  and  Experimentation  Activities 


Mining  and 
exploration 

Development 

Refinement 

Maturation 

Confirmation 

Joint  concept  development  phases 


Source:  Joint  Forces  Command. 
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Appendix  II:  Additional  Information  on  Joint 

Concept  Development  and  Experimentation 

Activities 

Table  3  provides  additional  information  about  the  characteristics,  scale, 
and  frequency  of  these  and  other  associated  activities  and  experiments. 

Table  3:  Types  of  Activities  and  Experiments  Associated  with  Joint  Experimentation 

Type  of  activities  and 
experiments 

Characteristics 

Scale  and  frequency 

Workshop 

Brief,  intense  experimentation  program  used  to  discover 
and  generate  concepts.  Participants  identify  focus  areas 
and  research  topics.  Workshops  are  conducted  iive  and 
may  have  virtual  components.  Subject  matter  experts 
participate. 

Small  scale 

Many  each  year 

Seminar 

Discussion  forums  used  to  discover  and  define  probiem 
boundaries,  pose  solutions,  and  exchange  information. 
Seminars  are  conducted  live,  and  may  have  virtual 
components.  Subject  matter  experts  participate. 

Small  scale 

Many  each  year 

War  game 

An  event,  using  simulated  and  emulated  systems,  centering 
on  the  interaction  of  two  or  more  opposing  forces  depicting 
a  possible  or  actual  scenario.  Identifies  key  variables  in  an 
experiment;  refines  concepts  and  assesses  alternatives. 

Scale  varies 

One  or  more  each  year 

Limited  objective  experiment 

An  event,  limited  in  scope,  that  permits  participants  to  see 
how  specific,  focused  ideas  or  concepts  will  work  in  a  given 
context  and  works  with  supporting  elements  of  the 
integrating  concept.  It  confirms  or  refines  experiments  and 
is  primarily  virtual  or  simulated.  May  include  workshops, 
seminars,  and  war  games. 

Small  scale 

Varies  (6  in  2001) 

Leveraged  event 

Service  or  other  organization’s  event  in  which  the  Joint 
Forces  Command  and  its  partners  share  lessons  learned. 
The  event  may  involve  the  use  of  live  forces  in  a  theater 
scenario. 

Scale  varies 

Frequency  varies 

Spiral 

A  group  of  smaller  events  and  meetings  requiring  live 
participants;  concepts  are  refined  and  plans  are  made  for 
larger  live  Joint  Forces  Command  experiments. 

Small  to  medium  scale 

A  few  each  year 

Major  simuiation  experiment 

A  constructive,  virtual,  or  live  experiment  that  makes 
extensive  use  of  models  and  simulation  to  provide  detailed 
information  on  the  expected  outcomes  of  an  event  or 
sequence  of  events.  It  may  use  live  and/or  simulated 
participants. 

Large  scale 

Not  more  than  one  every  2  years 

Major  field  experiment 

Integrates  several  smaller,  previous  events.  Determines 
how  effectively  the  concepts  previously  explored  in  smaller 
experiments  will  operate  when  integrated  with  the  concepts 
of  the  other  services  and  combatant  commands.  It  validates 
concepts  and  requires  live  participants  as  well  as  simulated 
opponents. 

Large  scale 

Not  more  than  one  each  year 
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Appendix  III:  Comments  from  the 
Department  of  Defense 


Report  number  is  now: 
GAO-02-856. 


POLICY 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

2000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-2000 


August  8, 2002 


Mr.  Heiuy  L.  Hinton,  Jr. 

Managing  Director,  Defense  Capabilities 
and  Management 
U.S.  General  Accounting  Office 
Washington,  DC  20548 

Dear  Mr.  Hinton: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report, 
GAO-02-853,  “Military  Transformation:  DOD  Actions  Needed  to  Better  Manage  Its 
Joint  Experimentation  Program,”  (GAO  code  350120).  DoD  concurs  with  the  report  and 
the  GAO’s  overall  assessment.  The  enclosure  contains  the  Department’s  response  to 
GAO  recommendations.  The  Department  provided  the  GAO  separately  the  substantive 
and  general  comments  with  supporting  rationale. 

In  general,  the  Department  believes  that  the  GAO  findings  are  thoroughly 
researched  and  accurately  reflect  both  the  history  and  the  progress  that  the  Department 
has  made  towards  improving  the  joint  experimentation  process.  The  report  is  fair  and 
balanced  in  that  it  acknowledges  the  scope  of  the  task  of  transformation  and  the  critical 
importance  of  the  supporting  experimentation  process.  The  Department  is  aggressively 
pursuing  actions  that  will  transform  our  military  forces.  Roles  and  responsibilities  for 
implementing  the  Department’s  transformation  strategy,  including  joint  experimentation, 
will  be  clarified  in  the  Department’s  Transformation  Plaiming  Guidance,  currently  in 
staffing. 


The  Department  appreciates  the  opportunity  to  review  and  comment  on  the  draft 
GAO  report. 


Sincerely, 


Chn^O] 
Deputy  Assdsti 
For  Plans  and 


ler  JyLamb 
Tetary  of  Defense 
(Acting) 


w/enclosure 
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Appendix  III:  Comments  from  the  Department 
of  Defense 


Now  on  p.  25. 


Now  on  p.  25. 


Now  on  p.  26. 


GAO  DRAFT  REPORT  DATED  JULY  11,  2002 
(GAO  CODE  350120) 

"MILITARY  TRANSFORMATION:  DOD  ACTIONS  NEEDED  TO  BETTER  MANAGE  ITS  JOINT 
EXPERIMENTATION  PROGRAM” 

DEPARTMENT  OF  DEFENSE  COMMENTS  TO  THE  GAO  RECOMMENDATIONS 

RECOMMENDATION  1:  To  improve  the  management  of  DoD’s  joint  experimentation 
program,  the  GAO  recommended  that  the  Secretary  of  Defense  direct  the  Chairman  of  the  Joint 
Chiefs  of  Staff  to: 

•  approve  and  issue  guidance  that  clearly  defines  the  information  required  to  accompany 
joint  experimentation  recommendations  for  the  Joint  Requirements  Oversight  Council’s 
review  and  approval;  and 

•  require  the  Commander  in  Chief  of  the  U.S.  Joint  Forces  Command  to  develop  strategic 
planning  tools  to  use  in  managing  and  periodically  assessing  the  progress  of  its  joint 
experimentation  program,  (p.  27/GAO  Draft  Report) 

DOD  RESPONSE:  Concur  with  the  problems  identified.  The  Transformation  Planning 
Guidance  (TPG),  currently  in  the  staffing  process,  or  subsidiary  guidance  documents 
identified  in  the  TPG,  will  address  and  clarify  both  the  JROC  process  issue  and  the 
requirement  for  the  implementation  of  strategic  planning  tools  at  JFCOM. 

RECOMMENDATION  2:  The  GAO  recommended  that  the  Secretary  of  Defense  develop  both 
quantitative  and  qualitative  performance  measures  for  joint  experimentation  in  DoD’s  annual 
performance  report  to  provide  a  better  assessment  of  the  program’s  contribution  to  advancing 
military  transformation,  (p.  27/GAO  Draft  Report) 

DOD  RESPONSE:  Concur  with  the  recommendation.  The  Transformation  Planning 
Guidance,  currently  in  the  staffing  process,  or  subsidiary  guidance  documents  identified  in 
the  TPG,  will  address  this  issue  by  directing  JFCOM  to  revise  its  Concept  Development  and 
Experimentation  Campaign  Plan. 

RECOMMENDATION  3:  The  GAO  recommended  that  the  Secretary  of  Defense  clarify  the 
role  of  the  Force  Transformation  Office  and  its  relationship  to  the  Chairman  of  the  Joint  Chiefs 
of  Staff,  the  Joint  Forces  Command,  and  other  key  DoD  stakeholders  in  DoD  ’  s  j  oint 
experimentation  program,  (p.  27/GAO  Draft  Report) 

POP  RESPONSE:  Concur  with  the  recommendation.  The  report  discusses  several  problems 
and  issues  with  the  joint  experimentation  program.  In  general  these  relate  to  the  need  to 
clarity  of  roles  and  missions  across  DOD,  the  implementation  of  change  recommendations, 
the  necessity  for  strategic  management  tools,  and  the  requirement  for  meaningful 
performance  measures.  The  Transformation  Planning  Guidance  will  address  these  issues  by 
clarifying  roles  and  missions  across  the  Department,  implementing  change  recommendations, 
and  establishing  clear  objectives.  When  coordinated  and  implemented,  the  TPG  will  ensure 
that  all  stakeholders  are  prepared  to  respond  positively  to  the  observations  and 
recommendations  listed  in  the  GAO  report. 


1  Enclosure  (1) 
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Appendix  IV:  Organizations  and  Offices 
Contacted 


Washington,  D.C., 
Area 


The  Joint  Staff 

•  Operational  Plans  and  Interoperability  Directorate 
•  Joint  Vision  and  Transformation  Division 

•  Command,  Control,  Communications,  and  Computers  Directorate 

•  Force  Structure,  Resources,  and  Assessment  Directorate 

Department  of  the  Army 

•  Directorate  of  Training 

•  Directorate  of  Integration 

•  Directorate  for  Strategy,  Concepts,  and  Doctrine 

Department  of  the  Navy 

•  Office  of  the  Deputy  Chief  of  Naval  Operations  for  Warfare 
Requirements  and  Programs 

•  Marine  Corps  Combat  Development  Command 

Department  of  the  Air  Force 

•  Wargaming  and  Experimentation  Division 
Other  Organizations 

•  Booz  Allen  Hamilton 

•  The  Carlyle  Group 

•  Center  for  Strategic  and  Budgetary  Assessments 

•  Hicks  &  Associates,  Inc. 


Department  of  Defense 

•  Office  of  the  Secretary  of  Defense,  Program  Analysis  and  Evaluation 

•  Office  of  the  Under  Secretary  of  Defense  for  Policy 

•  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics 

•  Joint  Advanced  Warfighting  Program 

•  Defense  Advanced  Research  Project  Agency 

•  Office  of  Force  Transformation 
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Appendix  IV:  Organizations  and 
Offices  Contacted 


Honolulu,  Hawaii, 

Area 

U.S.  Pacific  Fleet 

•  Joint  Experimentation  Office 
U.S.  Army  Pacific 

•  Internal  Review  Office 
U.S.  Pacific  Air  Forces 

•  Operational  Requirements 
Marine  Forces  Pacific 


U.S.  Pacific  Command 

•  Joint  Experimentation  Division 

•  Intelligence  Architecture  and  Systems  Division 

•  Exercise  Division 

•  Regional  Strategy  and  Policy  Division 


Miami,  Fla.,  Area 


U.S.  Southern  Command 

•  Future  Operations 

•  Programs  and  Resources  Directorate 

•  Office  of  Analysis  and  Simulations 


Newport,  R.L,  Area 


Navy  Warfare  Development  Command 


Norfolk,  Va.,  Area 


U.S.  Joint  Forces  Command 

•  Joint  Experimentation  Directorate 

•  Joint  Integration  and  Interoperability  Directorate 
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Appendix  IV:  Organizations  and 
Offices  Contacted 


Department  of  the  Army 

•  Training  and  Doctrine  Command 

•  Army  Joint  Venture  Directorate 

•  Army  Audit  Agency 

Department  of  the  Air  Force 

•  Air  Combat  Command 

•  Air  Force  Experimentation  Office 

Joint  Forces  Staff  College 


Tampa,  Fla.,  Area 


U.S.  Central  Command 

•  Futures  Division 

•  Exercise  Division 

U.S.  Special  Operations  Command 

•  Special  Operations  Requirements  and  Resources 

•  Joint  Experimentation  Division 

•  Special  Operations  Acquisition  and  Logistics 

•  Intelligence  and  Information  Operations 
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GAO  Contacts 


Sharon  L.  Pickup  (202)  512-9619 
Richard  G.  Payne  (757)  552-8119 
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GAO’S  Mission 

The  General  Accounting  Office,  the  investigative  arm  of  Congress,  exists  to 
support  Congress  in  meeting  its  constitutional  responsibilities  and  to  help 
improve  the  performance  and  accountability  of  the  federal  government  for  the 
American  people.  GAO  examines  the  use  of  public  funds;  evaluates  federal 
programs  and  policies;  and  provides  analyses,  recommendations,  and  other 
assistance  to  help  Congress  make  informed  oversight,  policy,  and  funding 
decisions.  GAO’s  commitment  to  good  government  is  reflected  in  its  core  values 
of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost  is 
through  the  Internet.  GAO’s  Web  site  (www.gao.gov)  contains  abstracts  and  full- 
text  files  of  current  reports  and  testimony  and  an  expanding  archive  of  older 
products.  The  Web  site  features  a  search  engine  to  help  you  locate  documents 
using  key  words  and  phrases.  You  can  print  these  documents  in  their  entirety, 
including  charts  and  other  graphics. 

Each  day,  GAO  issues  a  list  of  newly  released  reports,  testimony,  and 
correspondence.  GAO  posts  this  list,  known  as  “Today’s  Reports,”  on  its  Web  site 
daily.  The  list  contains  links  to  the  full-text  document  files.  To  have  GAO  e-mail 
this  list  to  you  every  afternoon,  go  to  www.gao.gov  and  select  “Subscribe  to  daily 
E-mail  alert  for  newly  released  products”  under  the  GAO  Reports  heading. 

Order  by  Mail  or  Phone 

The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each.  A 
check  or  money  order  should  be  made  out  to  the  Superintendent  of  Documents. 
GAO  also  accepts  VISA  and  Mastercard.  Orders  for  100  or  more  copies  mailed  to  a 
single  address  are  discounted  25  percent.  Orders  should  be  sent  to: 

U.S.  General  Accounting  Office 

441  G  Street  NW,  Room  LM 

Washington,  D.C.  20548 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (202)  512-2537 

Fax:  (202)  512-6061 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Public  Affairs 

Jeff  Nelligan,  managing  director,  NelliganJ@gao.gov  (202)  512-4800 

U.S.  General  Accounting  Office,  441  G  Street  NW,  Room  7149 

Washington,  D.C.  20548 
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